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TUTORIAL 


AN  INDEX  TO  MEASURE  A 
SYSTEM'S  PERFORMANCE  RISK 


Paul  R.  Garvey  and  Chien-Ching  Cho 

Technical  Performance  Measures  (TPMs)  are  traditionally  defined  and  evaluated 
to  assess  how  well  a  system  is  achieving  its  performance  requirements.  Typically, 
dozens  of  TPMs  are  defined  for  a  system.  Although  they  generate  useful 
information  and  data  about  a  system’s  performance,  little  is  available  in  the 
program  management  community  on  how  to  integrate  these  measures  into  a 
meaningful  measure  of  the  system’s  overall  performance  risk.  This  paper 
presents  how  individual  TPMs  may  be  combined  to  measure  and  monitor  the 
overall  performance  risk  of  a  system.  The  approach  consists  of  integrating 
individual  technical  performance  measures  in  a  way  that  produces  an  overall 
risk  index. The  computed  index  shows  the  degree  of  performance  risk  presently 
in  the  system.  It  identifies  risk-driving  TPMs,  enables  monitoring  time-history 
trends,  and  reveals  where  management  should  target  strategies  to  lessen  or 
eliminate  the  performance  risks  of  the  system. 


As  a  system  evolves  through  its  ac¬ 
quisition  and  deployment  phases, 
management  defines  and  derives 
measures  that  indicate  how  well  the  sys¬ 
tem  is  achieving  its  performance  require¬ 
ments.  These  measures  are  known  as 
Technical  Performance  Measures 
(TPMs)  (Blanchard  &  Fabry cky,  1990; 
Department  of  Defense  [DoD],  2002). 
Measures  such  as  Weight,  Mean-Time-Be- 
tween-Failure,  and  Detection  Accuracy 
are  among  the  types  of  TPMs  often  de¬ 
fined  on  programs.  Technical  perfor¬ 
mance  measurements  can  be  taken  from 
a  variety  of  sources.  This  includes  data 
from  system  testing,  system  simulations, 
and  experimentation.  Depending  on  the 


source  basis  for  these  data,  and  the  devel¬ 
opment  phase  of  the  system,  performance 
data  may  be  derived  from  a  mix  of  actual 
or  forecasted  values. 

The  program  management  community 
has  little  in  the  way  of  methodology  for 
quantifying  performance  risk  as  a  func¬ 
tion  of  a  system’s  individual  technical  per¬ 
formance  measures.  The  approach  pre¬ 
sented  herein  consists  of  computing  a  risk 
index  derived  from  these  individual  per¬ 
formance  measurements.  The  index  shows 
the  degree  of  performance  risk  presently 
in  the  system,  supports  identifying  risk¬ 
driving  TPMs,  and  can  reveal  where 
management  should  focus  on  improv¬ 
ing  technical  performance  and,  thereby, 
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lessen  risk.  When  the  index  is  continu¬ 
ously  updated,  management  can  moni¬ 
tor  the  time-history  trend  of  its  value.  This 
enables  management  to  assess  the  effec¬ 
tiveness  of  risk  reduction  actions  being 
targeted  or  achieved  over  time. 

In  general,  TPMs  are  measures  that, 
when  evaluated  over  time,  must  either 
decrease  to  meet  a  system’s  perfor¬ 
mance  requirements  or  increase  to  meet 
performance  requirements.  Thus,  each 
TPM  can  be  assigned  to  one  of  two  cat¬ 
egories.  For  this  paper,  define  Category 
A  as  the  collection  of  TPMs  whose  val¬ 
ues  must  decrease  to  achieve  a  system’s 
threshold  performance  requirements. 
Define  Category  B  as  the  collection  of 
TPMs  whose  values  must  increase  to 
achieve  a  system’s  threshold  perfor¬ 
mance  requirements.  This  is  illustrated 
in  Figure  1. 

In  Figure  1,  the  horizontal  axis  repre¬ 
sents  measurement  date.  This  is  the  date 
when  the  actual  or  forecasted  value  of  the 
TPM  was  taken  or  derived.  The  vertical 
axis  represents  the  value  of  the  TPM  at 
the  corresponding  measurement  date. 


In  Figure  1,  Vthres  denotes  the  threshold 
performance  value  for  the  TPM.  This  is 
the  minimum  acceptable  value  for  the 
TPM.  It  marks  the  boundary  between  the 
regions  of  acceptable  versus  unaccept¬ 
able  performance  risk. 

It  is  assumed  that  TPMs  defined  for  a 
system  are  done  judiciously;  that  is,  only 
those  TPMs  truly  needed  to  properly  mea¬ 
sure  a  system’s  overall  technical  perfor¬ 
mance  are  defined,  measured,  and  moni¬ 
tored.  Given  this,  acceptable  performance 
risk  can  be  defined  as  the  condition  when 
all  TPMs  reach,  or  extend  beyond,  their 
individual  threshold  performance  values. 
Conversely,  unacceptable  performance 
risk  can  be  defined  as  the  condition  when 
one  or  more  TPMs  have  not  reached  their 
individual  threshold  performance  values. 

A  Performance  Risk  Index  Measure 

The  following  presents  an  index  de¬ 
signed  to  measure  the  performance  risk 
of  a  system.  The  index  provides  a 
numerical  indicator  of  how  well  a 
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Figure  1.  Category  A  and  Category  B  Technical  Performance  Measures 
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developing  system  is  progressing  toward 
its  threshold  performance  requirements. 
It  serves  as  a  yardstick  that  enables  man¬ 
agement  to  measure  the  “distance”  the 
system  is  from  its  minimum  performance 
thresholds  and  to  monitor  trends  over 
time. 


To  develop  the  risk  index,  it  is  neces¬ 
sary  to  normalize  the  TPM  “raw”  values 
into  a  common  and  dimensionless  scale. 
Figures  2  and  3  show  such  scales  for 
Category  A  and  Category  B  TPMs.  In 
these  figures,  the  left-most  vertical  scales 
reflect  TPM  raw  values  (their  native 
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Figure  2.  Normalized  Category  A  TPM 
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units)  taken  from  engineering  measure¬ 
ments,  tests,  experiments,  or  prototypes. 
The  right-most  vertical  scales  reflect  TPM 
normalized  values.  Here,  threshold  values 
are  all  normalized  to  one.  This  scale  trans¬ 
formation  is  done  for  each  TPM  in  each 
category.  This  allows  management  to  com¬ 
pare  the  progress  of  each  performance 
measure  in  a  common  and  dimensionless 
scale.  From  these  normalized  scales,  an 
overall  measure  of  the  extent  to  which  the 
performance  of  the  system  meets  its  thresh¬ 
old  requirements  can  then  be  determined. 
Next  are  formulas  to  derive  this  measure. 
This  is  followed  by  a  computation  example 
to  illustrate  the  application  context. 

As  mentioned  previously,  let  Category 
A  be  the  set  of  TPMs  that  need  to  be  re¬ 
duced  to  their  threshold  values.  In  Figure 
2,  let  Vti  ,\j  be  the  value  at  time  ti  for  the  /'th 
TPM  in  Category  A  and  Vthres  A/  be  the 
threshold  value  to  which  the  /th  TPM  is 
driven.  Define  vti  A/  to  be  a  normalized 
TPM  value  against  its  threshold  as  follows 
(assuming  both  V*  a/  and  VthreSj  Ay-  are  greater 
than  0): 

A.  A j  ~  max  { Vji  A p  Vthres,  A/}  ^  Vthres,  Aj 
(i.e.,  threshold  met  if  Vtij  A/  <  Vthres  a/) 

—  max{  Aj  I  Vthres,  Aj ^  1 } 

—  max{  ( Vfhres ,  Aj  ~  Vthres,  Aj  F  Efi,  Aj)  ! 

Vthres,  Aj ■>  1 } 

—  max  {  1  +  (  V)/,  ,,\j  —  Vfhres,  Aj)  ! 

Vthres,  Aj,  1}(>1)  Eqt  1 

Equation  1  is  the  formula  for  v„-  A  in  Fig¬ 
ure  2,  which  brings  out  the  overage  above 
1.  Similarly,  let  Category  B  be  the  set  of 
TPMs  that  need  to  be  increased  to  their 
threshold  values.  In  Figure  3,  let  Vti,  Bk  be 


the  value  at  time  ti  for  the  Mi  TPM  in  Cat¬ 
egory  B  and  Vthres  Bk  be  the  threshold  value 
to  which  the  Mi  TPM  is  driven.  Define  v„  Bk 
to  be  a  normalized  TPM  value  against  its 
threshold  as  follows  (assuming  both  V),  Bk 
and  Vfhres,  Bk  arc  greater  than  0): 

vti,  BA:  —  min{  Vfj  Bk,  Vfhres ,  Bk  1  !  Vfhres,  Bk 

(i.e.,  threshold  met  if  Vti,  Bk  >  Vthres  M) 

—  min  { Vti  Bk  /  V^res,  ba^  )  1 

—  min  {  ( Vfhres,  BA  -  Vfhres,  Bk  F  V*  Bk) 

I  Vfhres,  Bk,  1  } 

—  min{  1  -  ( Vfhres ,  BA  "  V;L  ba)  I 

Vfhres,  Bk,  1}  (<  1)  Eqt  2 

Equation  2  is  the  formula  for  vti  Bk  in 
Figure  3,  which  brings  out  the  under¬ 
age  below  1.  From  the  normalized  val¬ 
ues,  we  now  calculate  their  average  dif¬ 
ference  from  1  for  each  category  and 
use  it  as  the  category’s  TPM  Risk  Index 
(TRI).  Assume  j  -  1,  2,  ...  ,  m  for  Cat¬ 
egory  A  (m  elements)  and  k  -  1,  2,  ...  , 
n  for  Category  B  (n  elements),  then 

TRIj;,  A  =  I  K',  At  ■  1)  +  K,  A2  -  1)  +  •  ■  • 

+  K,  Am  -  1)]  /  m 

-  IK,  A1  +  vti,  A2  +  •  •  •  +  vti,  Am)  / 

m]  -  1  Eqt  3 

TRI*  B  —  [(1  “  vti,  Bl)  +  (1  “  vti,  B2)  + 

+  (1  -  V*  Bn)l  /  n 

-  1  “  tK,  Bl  +  vti,  B2  F  ...  + 

Vti,  Bn)  /  n]  Eqt  4 

These  two  indices  show  the  aver¬ 
age  overage  or  underage  for  TPMs  in 
Category  A  or  Category  B  when  their 
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individual  threshold  values  are  re¬ 
scaled  to  1.  To  combine  all  normal¬ 
ized  values  into  an  overall  risk  index, 
we  first  convert  the  TPMs  in  Category  A 
into  equivalent  ones  in  Category  B.  This 
is  because  the  normalized  values  for  Cat¬ 
egory  A  can  differ  in  orders  of  magni¬ 
tude  from  those  for  Category  B  (e.g.,  1000 
vs.  0.5).  An  overall  index,  based  on  the 
normalized  values  as  calculated,  will  be 
unduly  influenced  by  large  values.  The 
result,  though  correct,  can  be  difficult  to 
interpret. 

To  make  such  a  conversion,  observe 
that  for  the  jth  TPM  in  Category  A  with 
value  Vti  ,\j  and  threshold  Vthres  A/,  an 
equivalent  TPM  in  Category  B  can  be 
constructed  with  value  Uti  A/  =  l/Vti  A/  and 
threshold  Uthres>  Aj  =  1  /VthreSy  Aj.  Typically, 
the  reciprocal  of  a  TPM  is  just  as  practi¬ 
cal.  For  example,  a  failure  rate  or  a  pro¬ 
cessing  delay  that  is  to  be  reduced  can  be 
taken  in  its  reciprocal  respectively  as  a 
mean  time  between  failure  or  a  comple¬ 
tion  rate  that  is  to  be  increased. 

The  probability  of  a  certain  undesirable 
event  (e.g.,  misclassification  or  an  error 
exceeding  the  tolerance)  or  unavailabil¬ 
ity  of  a  certain  desirable  state  (e.g.,  sys¬ 
tem  working  or  parts  in  hand)  are  more 
subtle.  But  their  reciprocals  can  be  viewed 
as  the  expected  number  of  events  that  will 
contain  one  such  undesirable  event  or  the 
expected  length  of  time  that  will  contain 
one  unit  time  of  such  a  desirable  state  being 
unavailable.  Although  their  complements 
(as  opposed  to  reciprocals)  can  also  be 
used  as  Category  B  TPMs,  it  is  not  recom¬ 
mended  as  the  complements  are  usually 
close  to  1  and  their  further  improvements 
toward  1  do  not  show  much  difference 
when  normalized. 


The  normalized  value  for  a  Category 
A  TPM  converted  into  a  Category  B  TPM 
is,  by  definition 

Ufi,Aj  ~  rtlin{  Ufj  Aj,  Ufhres.Aj}  !  U fores,  Aj 

—  min{  1/Vfi.  Aj,  1  /kj/jeei,  a/}  /  (1/ 
Vthres,  Aj) 

—  [1  /  max{Vti  Aj,  Vt/ires  Aj})  /  (1/ 

Vthres,  Aj) 

—  1  /  [max  { Vtf'  Aj,  Vfores  Aj }  / 

Vthres,  Ay] 

=  1  /  vtii  Aj  (<  1 )  Eqt  5 

We  can  now  treat  all  TPMs  as  being  in 
Category  B  and  then  derive  an  overall  risk 
index. 

Let  TRI*fi  a  =  1  -  \(uti.  At  +  uti,  A2  +  •  ■  ■  + 
Uti,  Am)  /  m]  Eqt  6 

TRIfj,  B  =  1  -  [{vti  B1  +  vti,  B2+  ■  ■  •  + 
vti,  Bn)  /  n]  as  before 

Eqt  7 

then  TRI„  An  =  1  -  \{uti  A1  +  uti  A2  +  ■  ■  ■  + 

uti,  Am  +  vti,  B1  +  vti,  B2  + 
■■■  +  vtii  Bn)/(m  +  n)] 

=  l-[(m(l-TRI\A)  + 
n(l  -  TRI„  B))  /  (m  +  n)] 

=  [m(TRFs  a)  +  n(TRI„  B)|  / 
(m  +  n)  Eqt  8 

where  TRI„  A11  is  the  overall  TPM  Risk 
Index  for  the  system,  computed  across 
all  of  the  system’s  TPMs.  Finally,  a  non¬ 
negative  weight  w Aj  could  be  assigned 
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to  (1  -  uti  A/)  for  the  jth  TPM  in  Cat¬ 
egory  A  and  wBk  to  (1  -  v„-  Bk)  for  the 
Ath  TPM  in  Category  B  (as  opposed  to  all 
having  an  equal  weight,  as  assumed  in  the 
discussion  above).  In  that  case,  it  can  also 
be  shown  that 

TRI\  A  =  1  -  [(wA1ms,  At  +  wA2Mri,  A2  +  •  ■  ■ 
",Am%,  Am)  ^  i 

Eqt  9 

where  WA=  wA1  +  Wa2  +  ■  ■  -  +  wAm 
TRIri,  B  -  1  ■  [(WB  Ari,  B1  +  wB2vri,  B2+--- 

+  wBnvri,Bn)/WB] 

Eqt  10 

where  WB  -  wB1  +  wB2+  . . .  +  wBn 

and  TRI,  M  =  [WaTRI*b,a+WbTRIs,  b]  /  W 

Eqt  11 

where  W=  WA+WB 

Thus,  equation  1 1  is  the  most  general 
form  of  the  system’s  overall  TPM  Risk  In¬ 
dex. 

From  the  above,  note  that  TRI*„  A,  TRIn 
B,  and  TRIr/  A11,  equally  or  unequally 
weighted,  are  all  bounded  by  0  and  1.  A 
value  of  0  for  the  risk  indices  means  there 
are  no  unacceptable  risks  in  the  included 
TPMs,  each  achieving  (or  extending  be¬ 
yond)  its  threshold  value.  The  risk  indices 
can  be  asymptotically  near  1  and  that  im¬ 
plies  that  each  TPM  value  in  Category  A  is 
very  large  when  compared  to  its  threshold 
and/or  that  each  TPM  value  in  Category  B 
is  very  small  when  compared  to  its  thresh¬ 
old,  i.e.,  all  far  away  from  their  thresholds. 
When  the  TPMs  are  moving  toward  their 
thresholds,  the  risk  indices  are  moving  to¬ 
ward  0. 


Computation  Example  and 
Hme  History  Graph 


Suppose  Table  1  represents  a  system’s 
set  of  Category  A  and  Category  B  TPMs, 
along  with  their  hypothetical  threshold  and 
raw  values  for  six  measurement  dates. 
From  these  data,  what  is  the  system’s  over¬ 
all  technical  performance  risk  index?  How 
is  it  changing  over  time? 

From  the  data  in  Table  1  and  equations  9, 
10,  and  11,  we  can  derive,  for  each  mea¬ 
surement  date,  the  TPM  risk  indices  for  the 
Category  A  and  Category  B  TPMs,  as  well 
as  for  the  system’s  overall  TPM  Risk  Index. 
The  results  from  these  derivations  are  sum¬ 
marized  in  Table  2. 

Note  that  TRI  is  a  cardinal  measure.  This 
means  its  value  is  a  measure  of  the  “strength” 
or  “distance”  that  the  contributing  TPMs  are 
from  their  individual  threshold  performance 
values.  A  TRI  equal  to  0.5  is  truly  twice  as 
“bad”  as  one  equal  to  0.25. 

Figure  4  presents  a  time  history  trend  of 
the  TPM  risk  indices  for  the  data  in  Tables 
1  and  2.  Here,  the  trend  is  good.  All  three 
TRIs  are  heading  toward  0.  This  means  all 
TPMs  defined  for  the  system  are  converg¬ 
ing  toward  their  individual  threshold  per¬ 
formance  values.  In  practice,  management 
should  regularly  produce  a  graphic  sum¬ 
mary  such  as  this  to  monitor  the  extent  that 
each  risk  index  changes  over  time. 

Summary 


This  paper  provides  an  approach  and 
formalism  for  developing  an  overall  set 
of  quantitative  indices  that  measure  a 
system’s  performance  risk,  as  a  function 
of  its  TPMs.  Below  are  the  general  forms 
of  the  three  principal  risk  indices. 


194 


An  Index  to  Measure  a  System's  Performanre  Risk 


Table  1 .  A  Hypothetical  Category  A  and  Category  B  TPM  Data  Set 


CATEGORY  A  TPM 

Raw  Value 

Eqt  1 

Eqt  5 

Vthres,  A 

V(ti,  A) 

v(ti,  A) 

u(ti,  A) 

wt 

Measurement  Date  tl 

Average  Processing  Delay  (msecs) 

1.000 

3.000 

3.000 

0.333 

1.000 

Mean  Time  to  Repair  (mins) 

10.000 

50.000 

5.000 

0.200 

1.000 

Payload  Weight  (lbs) 

950.000 

2112.000 

2.223 

0.450 

1.000 

Time  for  Engagement  Coordination  (sec) 

0.010 

0.100 

10.000 

0.100 

1.000 

TRI*(t1 ,  A) 

0.729 

Eqt  9 

Measurement  Date  t2 

Average  Processing  Delay  (msecs) 

1.000 

2.860 

2.860 

0.350 

1.000 

Mean  Time  to  Repair  (mins) 

10.000 

43.000 

4.300 

0.233 

1.000 

Payload  Weight  (lbs) 

950.000 

1764.000 

1.857 

0.539 

1.000 

Time  for  Engagement  Coordination  (sec) 

0.010 

0.040 

4.000 

0.250 

1.000 

TRI*(t2,  A) 

0.657 

Eqt  9 

Measurement  Datet3 

Average  Processing  Delay  (msecs) 

1.000 

1.180 

1.180 

0.847 

1.000 

Mean  Time  to  Repair  (mins) 

10.000 

43.000 

4.300 

0.233 

1.000 

Payload  Weight  (lbs) 

950.000 

1328.000 

1.398 

0.715 

1.000 

Time  for  Engagement  Coordination  (sec) 

0.010 

0.032 

3.200 

0.313 

1.000 

TRI*(t3,  A) 

0.473 

Eqt  9 

Measurement  Date  t4 

Average  Processing  Delay  (msecs) 

1.000 

1.090 

1.090 

0.917 

1.000 

Mean  Time  to  Repair  (mins) 

10.000 

27.000 

2.700 

0.370 

1.000 

Payload  Weight  (lbs) 

950.000 

1189.000 

1.252 

0.799 

1.000 

Time  for  Engagement  Coordination  (sec) 

0.010 

0.020 

2.000 

0,500 

1.000 

TRI*(t4,  A) 

0.353 

Eqt  9 

Measurement  Date  t5 

Average  Processing  Delay  (msecs) 

1.000 

1.030 

1.030 

0.971 

1.000 

Mean  Time  to  Repair  (mins) 

10.000 

12.000 

1.200 

0.833 

1.000 

Payload  Weight  (lbs) 

950.000 

1008.000 

1.061 

0.942 

1.000 

Time  for  Engagement  Coordination  (sec) 

0.010 

0.010 

1.000 

1.000 

1.000 

TRI*(t5,  A) 

0.063 

Eqt  9 

Measurement  Date  t6 

Average  Processing  Delay  (msecs) 

1.000 

0.980 

1.000 

1.000 

1.000 

Mean  Time  to  Repair  (mins) 

10.000 

9.000 

1.000 

1.000 

1.000 

Payload  Weight  (lbs) 

950.000 

948.000 

1.000 

1.000 

1.000 

Time  for  Engagement  Coordination  (sec) 

0.010 

0.010 

1.000 

1.000 

TRI*(t6,  A) 

0 

Eqt  9 

(continued  on  page  196) 
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Table  1 .  A  Hypothetical  Category  A  and  Category  B  TPM  Data  Set 

(continued) 


CATEGORY  B  TPM 

Vthres,  B 

Raw  Value 
V(ti,  B) 

Eqt  2 
v(ti,B) 

wt 

Measurement  Date  tl 

Interceptors  Available  (no.  of  units) 

150.000 

67.000 

0.447 

1.000 

Mean  Time  Between  Failure  (hours) 

500.000 

100.000 

0,200 

1.000 

Single  Shot  Success  Probability  (%) 

0.950 

0.870 

0.916 

1.000 

Damage  Assessment  Accuracy  (%) 

0.995 

0.600 

0,603 

1.000 

Software  Coding  (no.  of  modules  coded) 

763.000 

578.000 

0.758 

1.000 

TRI(t1,  B) 

0.586 

EqtIO 

Measurement  Datet2 

Interceptors  Available  (no.  of  units) 

150.000 

128.000 

0.853 

1.000 

Mean  Time  Between  Failure  (hours) 

500.000 

189.000 

0.378 

5.000 

Single  Shot  Success  Probability  (%) 

0.950 

0.890 

0,937 

1.000 

Damage  Assessment  Accuracy  (%) 

0.995 

0.878 

0.882 

1.000 

Software  Coding  (no.  of  modules  coded) 

763.000 

643.000 

0.843 

1.000 

TRI(t2,  B) 

0.399 

EqtIO 

Measurement  Datet3 

Interceptors  Available  (no.  of  units) 

150.000 

134.000 

0.893 

1.000 

Mean  Time  Between  Failure  (hours) 

500.000 

223.000 

0.446 

5.000 

Single  Shot  Success  Probability  (%) 

0.950 

0.910 

0.958 

1.000 

Damage  Assessment  Accuracy  (%) 

0.995 

0.940 

0.945 

1.000 

Software  Coding  (no.  of  modules  coded) 

763.000 

687.000 

0.900 

1.000 

TRI(t3,  B) 

0.342 

EqtIO 

Measurement  Datet4 

Interceptors  Available  (no.  of  units) 

150.000 

139.000 

0.927 

1.000 

Mean  Time  Between  Failure  (hours) 

500.000 

348.000 

0.696 

5.000 

Single  Shot  Success  Probability  (%) 

0.950 

0.934 

0.983 

1.000 

Damage  Assessment  Accuracy  (%) 

0.995 

0.945 

0.950 

1.000 

Software  Coding  (no.  of  modules  coded) 

763,000 

698.000 

0.915 

1.000 

TRI(t4,  B) 

0.194 

EqtIO 

Measurement  Datet5 

Interceptors  Available  (no.  of  units) 

150.000 

142.000 

0.947 

1.000 

Mean  Time  Between  Failure  (hours) 

500.000 

379.000 

0.758 

5.000 

Single  Shot  Success  Probability  (%) 

0.950 

0.940 

0.989 

1.000 

Damage  Assessment  Accuracy  (%) 

0.995 

0.999 

1.000 

1.000 

Software  Coding  (no.  of  modules  coded) 

763.000 

723.000 

0.948 

1.000 

TRI(t5,  B) 

0.147 

EqtIO 

Measurement  Datet6 

Interceptors  Available  (no.  of  units) 

150.000 

159.000 

1.000 

1.000 

Mean  Time  Between  Failure  (hours) 

500.000 

521.000 

1.000 

5.000 

Single  Shot  Success  Probability  (%) 

0.950 

0.990 

1.000 

1.000 

Damage  Assessment  Accuracy  (%) 

0.995 

1.000 

1.000 

1.000 

Software  Coding  (no.  of  modules  coded) 

763.000 

763.000 

1.000 

1.000 

TRI(t6,  B) 

0 

EqtIO 
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Table  2. 

Technical  Performance  Measure  (TPM)  Risk  Index  Summaries 


Measurement 

Date 

TPM  Risk  Index  for 
Category  A  TPMs 
TRI\a 

Eqt  9 

TPM  Risk  Index  for 
Category  B  TPMs 
TRI(;,b 

Eqt  10 

Overall  TPM  Risk  Index 
for  the  System 

TRIf/,  All 

Eqt  11 

tl 

0729 

0.586 

0.63 

t2 

0.657 

0.399 

0.478 

t3 

0.473 

0.342 

0.382 

t4 

0.353 

0.194 

0.243 

t5 

0.063 

0.147 

0.121 

t6 

0 

0 

0 

TRI  Value 


At  1 6  All  TPMs 
Reach  Thresholds 


Figure  4.  Illustrative  Technical  Performance  Measure  (TPM) 
Risk  Index  Time  History  Trend 
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Category  A: 

TRI**  A  “  1  “  I  ( w A I  un,  A1  +  WA2 uti,  A2  + 

. . .  +  WAmM„;  Am)  /  WA] 

where  WA  =  wA1  +  wA2  +  . . .  +  wAm 

Category  B: 

TRI(,  B  -  1  ■  [(WBlVft  B1  +  wB2vti,  B2  + 

■■■+  wBnvrl,  Bn)/WB] 

where  WB  =  wB1  +  wB2  +  . . .  +  wBn 

Overall  Risk  Index: 

TRIfi,  aii  =  [WaTRI\  a  +  WbTRI„-  b]  /  W 

where  W  =  WA  +  WB 

To  conclude,  key  features  of  the  ap¬ 
proach  presented  in  the  paper  are  sum¬ 
marized  as  follows: 

•  Provides  Integrated  Measures  of 
Technical  Performance:  This  ap¬ 
proach  provides  management  with  a 
way  to  transform  the  typically  dozen 
or  more  TPMs  into  common  mea¬ 
surement  scales.  From  this,  all  TPMs 
may  then  be  integrated  and  combined 
in  a  way  that  provides  management 
with  meaningful  and  comparative 


measures  of  the  overall  performance 
risk  of  the  system,  at  any  measure¬ 
ment  time  t. 

•  Measures  Technical  Performance 
as  a  Function  of  the  Physical  Pa¬ 
rameters  of  the  TPMs:  This  ap¬ 
proach  operates  on  actual  or  pre¬ 
dicted  values  from  engineering  mea¬ 
surements,  tests,  experiments,  or  pro¬ 
totypes.  As  such,  the  physical  param¬ 
eters  that  characterize  the  TPMs  pro¬ 
vide  the  basis  for  deriving  the  TPM 
risk  indices. 

•  Measures  the  Degree  of  Risk  and 
Monitors  Change  over  Time:  The 

computed  TPM  risk  indices  show  the 
degree  of  performance  risk  that  pres¬ 
ently  exists  in  the  system,  supports 
the  identification  and  ranking  of  risk¬ 
driving  TPMs,  and  can  reveal  where 
management  should  focus  on  im¬ 
proving  technical  performance  and, 
thereby,  lessen  risk.  If  the  indices  are 
continuously  updated,  then  manage¬ 
ment  can  monitor  the  time-history 
trends  of  their  values  to  assess  the  ef¬ 
fectiveness  of  risk  reduction  actions 
being  targeted  or  achieved  over  time. 
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